Online Action Detection

In the computer vision problem of online action detection, the goal is to detect the start of an action in a video stream as soon as it happens. For instance, if a child is chasing a ball, an autonomous car should recognize what is going on and respond immediately. This is a very challenging problem. First, only partial actions are observed. Second, there is a large variability in negative data. Finally, in real world data, large within-class variability exists. This problem has been addressed before, but only to some extent. First, we introduce a realistic dataset composed of 27 episodes from 6 popular TV series. The dataset spans over 16 hours of footage annotated with 30 action classes, totaling 6,231 action instances. Second, we analyze and compare various baseline methods with a newly-introduced evaluation protocol, showing this is a challenging problem for which none of the methods provides a good solution.